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ABSTRACT 

This report discusses results of a study that sought 
to determine if medical students perform equally on undergraduate and 
postgraduate academic measures regardless of the clinical site of 
their clerkship. A review of the literature revealed that 
accreditation requirements and sound education practices require that 
clinical sites be comparable with regard to medical education 
experience and student performance. The current study examined 
performance comparability for two clerkships (medicine and surgery) 
at a variety of sites using both academic measures (final clerkship 
percent score, National Board of Medical Examiners (NBME) Part 2 
subtest, and NBME Part 2 total) and postgraduate academic measures 
(NBME part 3 total and supervisor rating of first-year residents). It 
was found that virtually no differences in academic performance were 
found among sites for the two clerkships. The findings arc discussed 
in the context of methofjs for monitoring comparability of clerkship 
sites. Contains 32 references. (GLR) 
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Abstract 



Aocrecfitatton rec^irements and sound echication pr^ices require that clinical sites be 
comparable with regard to medlcai education experience and student performance. Past 
stiASes have generally found this to be the case for undergraduate ett:«tomic measures in 
imSvidual clerkships at mec^ sdioote with univer^ hospitals. The current study 
examined perfonnam con^rablHty for tm clerkships (medteine and surgery) in a 
community-based mecfical sch(K)l using both mecScal school acactemic measures (final 
derkship percent score. NBME Part 2 si^st, arKi NBME Part 2 total) and postgraduate 
ffi:»tomlc measures (NBME Part 3 total and supenHsor rating of first-year resident). Virtually 
no differences In academk: performance were found amor^ sites for the two clerkships. The 
firKfir^s are discussed in the context of methocte for monitoring comparability of clerkship 
sites. 
Purpose 

Chains in health care delivery systems over the last two dec^s have prompted 
medk»l sdK>ols whidi previously condiK^ cHnk»l motion exclusively in a university- 
based hosf^ to extend students' cSnksal education experierK:es into community sites. The 
newer, community-based n ectol sdmis have used this regional model since their 
Inception. Thus, for most medcal sdi(x>ls this <tovelof: ment has r^sed the program 
evakjation issue of comparablBty among clln^l sites. The research hypothesis for the study 
was: Do mec^ students perform equally on undergraduate and postgraduate academic 
measures regardless of the cHnical site of their cierl^ip? 
Review of LHerfltura 

Early reports described currk^lar variability among deri(ship sites and the need for 
comparable experiences In mecfical stuctont edu(»tion.i-3 Garraid and Verby^ found that 
medk»i students in a seven-month rural cHnteal experience had three times more patient 
encoumers; saw differsnt types of and were assign^ rTK>re responsibility for clinical 
proiilems; M more procedures; ami saw more female patients and more your^ patients but 
fewer age 41-65 year oM patients than their university-t^ised peers. Friedman et al.^ 
compared ec&ication experiences in an ot^etrics/gynecok>gy cieritship asKi found that 
Indlvkhjal community hospitals differed as much among themselves as from the university 
hospital. The unh^ersity hospital tended to provkfe a more academto clintoai experience and 
less active participation in patient care. Bomsztein and Julian^ verified the percent of 
required clinical activities completed by students at nine obstetrics/gynecoiogy ciericship sites 
and found that the mean percent completion rate varied from 71.5% to 98.3%. The 
completion rates for fult-time f^lty ^tes (86.6%) and private practtee sites (84.2%) were 
^milar. Strand et al.^ noted cOftorences in number of deliveries and student ratings of 
teaching effectiveness amoi^ eight obstetncs/gynecobgy deritship sites. Calhoun et ai.s 
examined a surgery cleritship at five different sites and four^ no differences In the total time 
alk>cated to patient care or education activities, but there were site variations for some of the 
12 specific F^ent care or education ^vities. Rosen et al.^ analyzed student togs to 
compare a univerelty hospital to five community hospitals with rsgard to diagnostic 
categories. Students saw more hematoiogy/oncolc^ and cardiovascular problems at the 
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university ho^>itai but more neuroiogic and enctocrine problems at the community hospital. 
Te^tor et al.^0 examined the srtivfties and types of (Mn\^ proUems seen at seven pecfiatric 
deri^hip sites rai^ng from exdusively inpa&ent expertonce to a near total outpatient 
experience. H(A surprisingiy. the ui^er^ and related hospital training locations haxi more 
structured teac^f^, tnteraction with reskients, and more severely ill patients, in contrast, 
community ou^[»atiem ^ngs offered greater amkHilatory patient opportunities, greater 
variety of cHnlcal problems, ami more of the comrron pwfiatric illnesses. Miller et al.^ ^ 
(tocumented cfifferences among six psychiatry dericsNp sites in a variety of dinical education 
festivities but reported that students saw no noteworthy «tonti^es anning the learning 
experiences m diftont sites. In Great Britain Walceford^^ found that peripheral (i.e., 
community) hospitals involved sHidents more in prEtt:tical procedures i»jt dd not differ from a 
teeing O.o.> university) hospital In exfiwrierHse with common mecficai conditions. Ktore 
encouragement and mors informal teaoMr^ occunvd at peripheral hosf^ls, and students 
wars generally more satisfied with their education experiences at tiiose sites. 

Stu(ies in a number of ^)edattiM have e)«mined medcal student perfomnance in a 
multi-site derkship: four in family prEKrtice, three in imemal mec^ne, four in ot^trk»/ 
gynecology, two in pecfiatrics, two in psychiatry, and five in surgery. Rabinowitzi3-i5 reported 
on the variaNHty in final examination perfdrmanoe (moffified essay questions) and student 
grad^ a! six Miy medidne derkship sites. Hot^ et al.i6 found that family medtolne clerks 
M not differ in final percent score in a comparison between rssktoncy teaching locations (6 
sites) and private group pr^^tees (5 sites). 

Calhoun et al.'i^ found that time altotted to different Internal medk:ine derkship 
»:^ities and level of re^nsiblHty a^n^ to the sUident varied among four hospitals, but 
the^ differences dkJ not correlate with Nattonai Board of Atocfical Examiners (NBME) Part 2 
mecSdne subtest scores. Ramsey et ai.i^ stucfied internal medksine clerits arKi found no site 
differences for a variety of objective measures. Whalen and Moses^® reported no differences 
for an internal mecHdne dericship in a comparison between two medk:al center locations and 
a combination of knjr community hospitals for subjective and final grades or an NBi^E 
medtoine shelf exam. 

irby et al.^ in a study of six obstetrics/gynecok>gy dericship sites found differences in 
the grsxtos for four witNnKSiericship criteria (a derfcship potest, presentation of an ob/gyn 
topte to peers, cUntoal perfonnance, and p£rtient write-ups). However, sites cSd not differ on 
oral examination scores or two external measures (okii/^n portion of a third-year 
comprehensive exam and NBME Pari 2 ob/gyn sut^st). In a later analysis of the same 
obstetrics/gynecoiogy clericship (but with eight rather than six sites) investigated by lrii>y et 
al.^, StrarKi et al.^ reported variabiHty among sites In oral exam performance tHit minimal 
site differences in written exam performance. Talbert et al.^i in a study of a university 
hospital and four community ho^>itals found that obstetric^gynecok>gy clerits dd not differ by 
site on NBME Part 2 or faculty cflnk^l ratings over a four year study period. Also, first 
po^graduate year performance of students entering obstetrics/gynecobgy resldendes did 
not differ when analyzed by obstetrics/ gynecoit^ deritship site. Bomsztein and Julian^ 
found no differences in cognitive examination perfomnance among nine obstetrics/ 
gynecotogy dericship sites. 

Joorabchi et al.^ investigated a pediatric clericship in three hospital settings 
(unb/ersity, county, private) and found differences in education activities, types and number of 
patients, and time allocation but found no differences in problem-solving and higher o^nitive 
level examinations. Similarly, in Israel Tamir et al.^s found that a university hospital and two 
nonuniversity hospitals differed in pediatric dintoai experiences, but students did not score 
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cKfforentty on their final pmMric exan^insiUon. However, students at nonuniversity hosfxtals 
were more sirtisfied with the qualty of their tr^nir^. 

Puiyear ami l/HNer^ i^crttied how deiks at one site of a psychiatry defkship 
(emergency room), where students historicaliy scored k)wer on a comprehensive final exam, 
improved their Bmm perfbnnance throi^h epedal attention from residents, m^r et ai.^ ^ 
reported small but statis^ly ^nificant differences among six psychiatry clerkship sites on 
written woric, cHnical work, and written examination. 

Schwartz et ai.^s sti^Sed ter: aurgeiy derk^ip sites (a unhfersity hospital and nine 
affiliated ho^pit^)- AlthCHigh stuctents were faiKkmily assigned to site and dki not cfiffer by 
site on a surgery |»ete^, there were some site ctifferenoes on four surgery subtests (general 
surgery, orthopecfics, urotoc^, and anesthestology). This firKttng was not surprisir^ given the 
many ho^)ital palrwise con^;)arisons across four Afferent subtests. J€K»bson et al.26 
compared a university setting with prhrate and county hospitate aruJ found no differences 
amoi^ surgery clerksMp students on an NBME shelf examinatton in surgery, mi 
exanMnatkm. or dntoal evakialton. McCarthy and King27 in a comparison of four hospital 
sites (university, county, private, and Veterans Administratkni) found no ciffersnces on a 
surgery knowtedge examinatton or achievement of procedural skills, levi^ et ai.28 studied 
two years of surgery derks at a univer^ and five community hosi^te and found no 
(fiffersnces on the ^rgery sul^st or total score on both a sdiool dlnteal comprehensive 
examination arKi NBME Part 2. Baciewka et al.» examined ttiree Afferent site combinations 
InvoMr^ a cfifferent mix of e}^r1ences during a three-month surgery deri«shlp and found no 
differences among sites in written (NBME-devetoped exam) and oral scors perfonnance. 

The cun^ stiKly extends previous inve^lgattons of comparat^Rty of education by 
examinif^ two multi-site ctorkships In a community-based medteal school (i.e., no university 
hosji^ as well as both medteal school and postgreKluate measures of performance. 

Methods ^ ... u. 

The internal medtelne and general surgery clerkships at Wright State University 
School of Medkiine (WSUSOM) wers exantined. At the time of the study, the intemal 
medk^ine clericship used prtndpaUy five different site comt^nations. A student spent six 
weeks at the first hospital and six weeks at the secorKi hosf^al. Now the cleri^ship uses four 
weeks at the first ho^ltal, four weeks at the ^cond hospital, and four weeks in an 
ambulatory setting. The surgery derkship laste e^ht weeks (six weeks in general surgery at 
one ^e and two weeks in a surgk^al subspedalty). The cun^nt stiKty investigatmi the six 
sites in whk^ the general surgery rotatton took p^ arKU the five priiKHpai intemal medteine 
site combinations. Student are assigned ranctomly to a site in the medk^ne clerkship, but in 
the surgery clericship student preferences for a site are aocommcKtated as much as possible. 

Sites (surgery) or site combinations (medteine} were compared on three medk»i 
school &caa9n^ variables (final deriishlp percent score, NBME Part 2 subtest, and NBME 
Part 2 total) and two pos^rac^e cK»demk; variables (NBME Part 3 total and total score on 
an 18-item measure of performance as a first-year resktont). 

The final dericship percent score In suigery is cateulated from three components: 
patient hlstoiy and physteal exam records written by the stwJent and presented to fanjity 
(20% of score), preceptor evaluation of student using cleriiship perfonnance form (40%), and 
objective and essay tests (40%). The final deriiship peicent score in meddne also has three 
elements: preceptor evakjation of the student usir^ deritship performance form (50%), oral 
exam based on patients written up by the student (10%), and objective quizzes and final 
examination (40%). This physidan supervisor rating of perfonnance as a first-year resident 
was devetoped at the University of Arizona.^ The intemal consistency reliability 
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(Cronbach^s alpha) of the physician supervisor rating form with eight WSUSOM classes is 

The Arizona form contains 16 scales: (a) 1 1 scales yield ratings from one to eight - 
interper^nal skills with sM. interpersonal skills with patients, fund of modk^aH knowiec^e, 
inotivatk>n^l^efK». vert»l and presentatton m\B, histoiy-taking skills, phystoal exan^nation 
skins, cfifferentia} cfic^nosis/problem-lisling sklHs. therapeutic pteuining, procec&jral skills, and 
overall evaluation; (b) seven scales yieM ratings from one to five - preparation at start of 
re^dency, resktonfs view of preparation, under^ncfing of goals ami objedives of residency, 
progress in learning and devek^ent durii^ first yecu* of reskiency, constructive use of 
fe6d)ack, use of educmkMi resources of the resktency program, and re^nsK^Kty for seif- - 
directed learning. The maximum score for the physk^an mipendsor rating of pos^raduate 
peiformanoe Is 123 (11 srales x 6 points -i- 7 scales x 5 points « 123). 

Qraduirtes of fCHir classes (1 988 through 1 991 ) were investigated (n > 349). Data 
were collect for 346 (99.7%) gr»&fites. Site comt^nation for 331 medk:ine clerkship 
sbidents was used in the analysis. Seventeen medicine ^rkship ^udents h£^ site 
oomt^nations other than tiie five principal k)cations. These site combinations were not 
induded cfeje to their small samphi size. Since the Class of 1 991 had not conned the first 
year of resktoncy and the Classes of 1^ throi^h 19^ h»j to grant pennission, fewer 
grs»&jates had data for the pomgraduate academk: variables (NBME Part 3 total, n « 163 of 
265, 62%; supervisor ratir^ of fir^ postgraduate year, n » 151 of 265. 57%). 

Univariate analysis of variance with site or site combination as the independent 
variable and each of the five SK^CKiemk: criteria as deperKtont measures was used. Tukey's 
multiple range test was used to foltow up significant ANOVA findings. Inferences were made 
at the .05 level of significance. 
Reauita 

Table 1 shows that in the medk^ine clerkship students differed by site combination for 
one undergraduate measure (clerkship score) but not for two other undergriKiluate measures 
(NBME Part 2 medk^ne subtest and NBME Part 2 total). Also, no site differences were found 
for tiie two postgrEK^e metres (NBME Part 3 total and supendsor rating as a first-year 
resktont). The sigrtificant ANOVA for ctorkship score was followed by Tukey's multiple range 
test. One ^e combinatton (SCI) cfiffered from one other site combir^on (SC2). Table 1 
shows that in tiie suipeiy clerkship there was one measure on whk;h sites varied: surgery 
clerkship score with two pairwise differences. 
Diseuaaion 

The Liaison Committee on Medial Educational has ^Mressed the importance of 
equivaierK^e or comparat^iity of ctorkship educatk>n among ^es: 

if required ctorkships in a single discipline are conducted in several 
hospitals, every effort must be msKto to er^re that the students 
receive equlvatont educational experiences, (p. 13) 
Our study c. nplements prevkHJS studies which analyzed cunicuiar content and education 
^vities and^or examined perfomiance.^^ Unlike other studies the Oinvnt investigation 
was conducted at a medk:al school without a university hospital arKU involve more than one 
clerkship. In addition, we used both undergraduate and postgraduate measures to assess 
equivalence. Kassebaum^^ has argued for tiie assessment of both types of outcomes in the 
evaluation of medical education programs. FurthemK>re, tiie use of a physician supervisor's 
rating of a graduated student when he/she is a first-year resktont answers to some extent 
Friedman et ai.'s^ concern that studies use 'clinical competence as the ultimate criterion 
measure." Also, our study used a cie.'kship that assigned students randomly and one which 
albwed student preferences. 
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For three m^isuies of mecficai school ecactomic performance and tw» measures of 
postgrac^^ accKtomic performance, our Interna} medicine and general surgery derfcships 
coffered tyy site only on clerkship score. For the final percent score in the internal mec^ne 
derkship one palnMse comparison of site comMnations was cfifferent by 3.2%. Similarly, we 
fourKi tm pairwise ^ cifferences for surgery deiteNp score with two sites higher than a 
third site a9% and 3.7%. The practk»l irr^rtanoe of these differences doei$ not seem 
gr^. and we can klentify no factors wl^wouUifficcHjntfCK^ the <ifferences. Our results add 
to those of other stucfies which have invest^ated student performance c^fferences among 
dsrk^ip sltes>7. 11. 13-29 

Wr^ht State University monitors compaiBt)iKty among sites for Its nine required 
derk^lps: emergency mecfidne, family prac^ce. mecficine, neurology. 
ot>stetrk:s/gyneook>gy. orthopeiScs. pedatr^, psychiatry, and surgery. Nine methods ara 
used to assure conipar^^ arnong education sites. These are: written ctorKship objectives, 
written performance expectatons for the ctorkship. written evaluation standards, review of 
patient togs maintained t)y ^udents, evakmtion of a ^>ecific numk>er of patient workups 
regardless of derksNp site, on-site fixity coordinator at each inmitutlon. r^ular meetings of 
the on-site coordinators, active paitteipmion t>y students in patient care, and c^se 
confererK»s for aH clerkship stwtonts at a central k>cation. These steps directed toward 
oomparc^Hty among sites may account in ^ for our lack of performance variak>IUty in the 
two clerkships reported. 



Table 1 

Comparison of site combinations for the internal medicine clerkship 
and sites for the general surge7 clerkship 



Clerkship score 
NBME Part 2 subtest 
NBME Part 2 total 
NBME Part 3 total 
Supervisor rating of 
first year reskient 



Internal ^todia'ne Clerkship 
F statistic p value 

3.70 .006* 
0.34 .848 
0.25 .907 
0.17 .952 



General Surgery Clerkship 



0.80 



.529 



3.66 
1.13 
1.71 
1.92 

1.59 



{LyalUfi 
.003** 
.343 

.132 
.093 

.166 



*Site combinatton 1 [SCI] (mean « 77.6%) was greater than SC2 (mean 74.4%). 
**Site 1 (mean a 83.8%) and Site 2 (mean « 83.6%) were greater than Site 3 (mean := 
79.9%). 
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